Conventional systems of computer arrays or clusters range from loosely coupled systems that communicate across Ethernet to systems including moderately coupled cores such as found in the symmetric multi-processing of multi-core Central Processing Unit (CPU) chips. These may be built with large instruction set CPUs with fast local cache memory supplementing shared Synchronous Dynamic Random Access Memory (SDRAM).
Large scale distributed model computing may include some level of synchronization such as Bulk Synchronous Parallel (BSP) processing, in which synchronization and communication are built into the model, as opposed to Parallel Random Access Machines (PRAM) in which synchronization and communication issues may be only minimally addressed. The BSP and PRAM processes cycle between a “concurrent computation” phase and a “communication” phase; with the BSP adding a “barrier synchronization” phase. Such cycles are referred to in the industry as a “superstep”.
Both BSP and PRAM architectures may have built in hardware, firmware, cores and protocols to assist and/or carry out some communication and synchronization operations, but software components generally take on responsibility for many communication and synchronization operations at a task level and/or Operating System (OS) level. Furthermore, a dynamic point-to-point communication network between processors is generally assumed to exist.
Problems that involve large amounts of processing time between synchronization steps can tolerate the low Input/Output (I/O) and synchronization properties of PRAM. Problems that involve frequent synchronization and I/O may generally use BSP or something better.